About

Dhirendra Singh is a highly skilled Data Engineer with over 7 years of experience in the IT industry. He is currently working as a Senior Big Data Developer at Mahindra Comviva, where he collaborates effectively with a distributed team to develop centric solutions/codes/queries. Dhirendra has a Bachelor of Technology degree in Information Technology from KIIT University and is certified in Agile Global Developer, Microsoft Azure Developer, and Apache Spark. He has extensive experience in analytics space and has demonstrated success in building data pipelines, data lakes, and providing insights from data to solve key business problems. Dhirendra is skilled in a range of technologies, including AWS, Python, Spark, Data Engineering, Big Data, Azure, Hadoop, GCP, Kafka, AWS Glue, Apache Airflow, HDFS, infra, and Apache Hive. He has worked on projects that involve PySpark, Spark Core, Spark RDD, Pair RDD, and Apache Spark Development. He is also experienced in designing and developing applications in Apache Spark using Python to compare the performance of Spark with other frameworks. In his previous roles, Dhirendra worked as a Senior Big Data Developer at Infosys and Wipro, where he developed PySpark codes to process raw data to prepared data using Spark and Python. He has also developed and executed Python codes to process AWS S3 raw data to AWS Redshift on AWS platforms. Additionally, he scheduled Apache Spark jobs using Dag Airflow to trigger PySpark codes/scripts automatically on a given time interval. Dhirendra has a strong engineering background and has worked on various projects involving data ingestion and handling clusters. He has also performed site reliability engineering responsibilities for Kafka platforms that scale 2 GB/Sec and 20 Million messages/sec. Overall, Dhirendra is a highly skilled Data Engineer with a passion for solving complex business problems using big data technology stack. He has a proven track record of delivering high-quality solutions and is an excellent communicator who can effectively communicate complex results.